Assessment of inter-rater and intra-rater reliability of the Luna EMG robot as a tool for assessing upper limb proprioception in patients with stroke—a prospective observational study

Background The aim of the study was to assess the inter-rater and intra-rater agreement of measurements performed with the Luna EMG (electromyography) multifunctional robot, a tool for evaluation of upper limb proprioception in individuals with stroke. Methods The study was conducted in a group of patients with chronic stroke. A total of 126 patients participated in the study, including 78 women and 48 men, on average aged nearly 60 years (mean = 59.9). Proprioception measurements were performed using the Luna EMG diagnostic and rehabilitation robot to assess the left and right upper limbs. The examinations were conducted by two raters, twice, two weeks apart. The results were compared between the raters and the examinations. Results High consistency of the measurements performed for the right and the left hand was reflected by the interclass correlation coefficients (0.996–0.998 and 0.994–0.999, respectively) and by Pearson’s linear correlation which was very high (r = 1.00) in all the cases for the right and the left hand in both the inter-rater and intra-rater agreement analyses. Conclusions Measurements performed by the Luna EMG diagnostic and rehabilitation robot demonstrate high inter-rater and intra-rater agreement in the assessment of upper limb proprioception in patients with chronic stroke. The findings show that Luna EMG is a reliable tool enabling effective evaluation of upper limb proprioception post-stroke.


INTRODUCTION
Proprioception is one of the senses making it possible to determine the position of one's body in space.It is an important part of the process involved in balance control, and in train the sensorimotor cortex (Olczak & Truszczy nska-Baszak, 2021;Zasadzka, Tobis & Trzmiel, 2020).
In the literature there are few reports focusing on the use of the Luna EMG robotic device in assessment and rehabilitation of upper extremity (Clinical Trials, 2024).One study investigated inter-rater and intra-rater reliability of Luna EMG in the assessment of upper limb proprioception; however, it was conducted in a group of healthy individuals (Leszczak et al., 2024).Therefore the current study was designed to examine inter-rater and intra-rater reliability of the Luna EMG multifunction robotic device as a tool intended to assess upper limb proprioception in patients with a stroke.

Type of study
The research was conducted in the form of a prospective observational study.
It was carried out in accordance with the ethical rules of the Declaration of Helsinki, and approved by the Local Bioethics Commission of the University of Rzeszow (resolution no.2022/036/W).Written consent was obtained from all the participants.The study was registered in the clinical trials register at ClinicalTrials.gov(registration number NCT05486052).

Participants
The minimum sample size was determined prior to the study, using a sample size calculator ("PLUS module" from Statistica 13.3 software).The procedure identified a population size of 77 patients with chronic stroke.Ultimately, 126 patients with chronic stroke were enrolled for the study.
The study was conducted in a rehabilitation and spa therapy hospital (physiotherapy laboratory) in the Podkarpackie Region, Poland.The inclusion criteria were defined as follows: completed first ischemic stroke; patient's informed, voluntary consent to participate; elementary (basic) gripping ability; the upper limb and hand paresis rated 4-5 on the Brunnström scale; degree of disability Rankin score of three; spastic tension of the upper limb and hand paresis not more than three on the modified Ashworth scale; and current health condition, confirmed by a medical examination, allowing participation in tests and exercises.The following exclusion criteria were applied: lack of the patient's informed, voluntary consent; second or subsequent ischemic stroke of the brain; haemorrhagic stroke; stroke of the brainstem and cerebellum; cerebellum disorders of the higher mental functions limiting the ability to understand and perform tasks during examinations; visual disturbances; mechanical and thermal injuries potentially impairing the hand grip function; concomitant neurological, rheumatological and orthopaedic diseases, including neuropathies that can affect proprioception, permanent contractures that may affect the grasping ability and locomotion; unstable medical condition (i.e., cardiopulmonary disturbances, consciousness disturbances; convulsions, sudden acute chest pain, cardiac arrhythmias, dyspnoea); metal implants; electronic implants; menstruation in women and epilepsy.
There were 200 patients with stroke in the rehabilitation and spa therapy hospital at the time of the study.Based on the inclusion and exclusion criteria, 147 patients were initially qualified to participate; however, after medical examination eight individuals were excluded.Ultimately 130 patients took part in Examination I, and 126 patients participated in Examination II (Fig. 1).

Procedure
The examinations were carried out using the diagnostic and rehabilitation device Luna EMG (EGZOTech Sp.z oo Gliwice, Poland).The assessments were carried out by two independent raters, under the same conditions and during the same time period.Two examinations of proprioception in relation to upper limb function were performed two weeks apart.The independent raters were physiotherapists with more than 5 years of experience working with people after stroke, trained in assessment, who had participated in previous published studies related to evaluation of the reliability of the Luna EMG Rehabilitation Robot to assess proprioception in the upper limbs in 102 healthy adults (Leszczak et al., 2024).
Luna EMG is a diagnostic and rehabilitation robotic device and its operation is based on reactive electromyography designed to train the sensorimotor cortex.The bioelectrical signals (EMG) obtained from the patient's muscles show that the movement is active (Rinderknecht et al., 2018;Oleksy et al., 2022;EGZOTECH, 2019).
Calibration of the Luna EMG device is performed annually by a trained service technician.This process includes a thorough validation of the measurement functions to ensure the accuracy and reliability of the device's readings.Regular calibration is essential for maintaining the device's performance and ensuring the consistency of test results over time.Before the start of the examination, the Luna EMG measured the weight of the limb to unweight it during the test procedure, thereby eliminating its impact on the patient's results.During the measurements, the patients remained in a sitting position, with the upper limb from the shoulder joint aligned with the trunk, and extended in the elbow, hips flexed at 90 .The trunk and the assessed limb were stabilised using straps.Each measurement comprised four tests performed in succession in each upper extremity.The range of motion was set between 0 and 90 degrees of elbow extension, which is limited by the device.The initial elbow joint position was 0 and the target position of elbow flexion was 60 ; the movement was performed either actively or passively (Leszczak et al., 2024) (Fig. 2).
During the passive movement phase, the speed of the device was set to 30 degrees per second for the learning process and 10 degrees per second for evaluating the target position.The maximal torque for the device was maintained at 10 Nm, and the holding time during the learning phase of the target position (elbow flexion 60 ) was set to 5 s.
Initially, the device passively conducted the patient's movement, then the patient was asked to "remember the movement/setting" during which time the equipment stopped for 5 s and the patient was tasked with learning to sense their position.Then, after 5 s, the limb passively returned to its initial position (elbow extension 0 ).The initial position was held for 3 s.After learning a passive movement, the patient was asked to actively perform the same movement with a stop: on the request, "please move your elbow to the position you learned and hold your arm in that position."All tests were performed with the patient's eyes closed.The duration of the study of one patient by two researchers was 30 min.

Statistical analysis
The analysis was carried out in the Statistica 13.3 program and the JASP program 0.18.3.0 (JASP Team (2024), Computer Software, Netherlands).The mean and standard deviation values were first calculated for each series of measurements, and then for the differences between the series of measurements compared in the study.The significance of differences in the average level of the two measurement series was assessed using the t-test for dependent samples in which no significant values should be observed, but it should be noted that this is not a key factor in assessing the agreement of measurements.A comparative analysis of the measurement series was performed using Pearson's linear correlation coefficient, as well as the key measure of agreement between two measurements, i.e., intraclass correlation coefficient (ICC).Bland-Altman method was applied as an alternative measure of agreement.
The value of p < 0.05 was assumed to reflect statistical significance.

Agreement between the measurements of right hand proprioception
As regards the assessment of the significance of differences, it was observed that Rater I on average recorded slightly higher measurement values in Exam I compared to those recorded in Exam II (p = 0.0001).A similar relationship was found between the measurements performed by the Raters in Exam I; on average the values recorded by Rater I were significantly higher compared to those identified by Rater II (p = 0.0220).The above observations were not confirmed by Pearson's linear correlation coefficients, which in all the analyses assessing both inter-rater and intra-rater agreement were very high (r = 1.00).The high agreement of the measurements carried out was also confirmed by intraclass correlation coefficients (ICC) (0.996-0.998) (Table 2).
The above findings are also confirmed by Bland-Altman plots.The highest mean deviation between measurements was identified in the assessment of agreement between Exam I and Exam II performed by Rater I (0.09 ), and in a considerable majority of the results the differences were up to 0.4-0.5 (Fig. 3).

Agreement between the measurements of left hand proprioception
In the assessments of left hand proprioception the findings show no significant differences between the mean results recorded by the two Raters (p = 0.4487; p = 0.4027); however, the mean results recorded by each Rater in the two Exams differed significantly (p = 0.0002;   Nevertheless, ultimately the agreement between the measurements was confirmed by high Pearson's linear correlation coefficients (r = 1.00) and ICC (0.994-0.999) (Table 3).By interpreting the results shown by the Bland-Altman plots for left hand proprioception measurement it was possible to determine that the highest level of agreement existed between Rater I and Rater II in Exam II.Mean deviation was only 0.013 whereas deviations in the specific values did not exceed +0.4 .The highest mean deviations were observed in measurements recorded by Rater II during Exam I and Exam II (0.13 ), with differences of nearly 2.5 in isolated cases.Despite the above observation, on the whole it can be said there is agreement between the measurements performed since a large majority of the differences in the results did not exceed 1.0 (Fig. 4).

DISCUSSION
The aim of the study was to assess the inter-rater and intra-rater agreement of measurements performed using the Luna EMG multifunctional robot as a tool for assessing upper limb proprioception in patients with stroke.The research has shown that Luna EMG is a reliable tool in the evaluation of upper limb proprioception.Measurements made with Luna EMG show high inter-rater and intra-rater reliability of the assessment of the proprioceptive sensation of the upper limb in individuals with stroke.
In the international literature there are no scientific reports investigating this subject matter, i.e., Luna EMG aided assessment of upper limb proprioception in patients with stroke.The only related studies published so far have presented evidence showing the reliability of this tool in healthy populations, when the device was applied to assess the sense of knee position (Oleksy et al., 2022) and to measure upper limb proprioception (Leszczak et al., 2024).The former study showed high reliability (ICC = 0.866-0.982) of Luna EMG aided assessments of both knee flexion and extension in active and passive modes in healthy individuals (Oleksy et al., 2022).Likewise, the latter study demonstrated a high consistency (ICC = 0.969-0.997) of upper limb proprioception measurements performed using the device (Leszczak et al., 2024).
In addition, the Cohen's d effect size was calculated.The Cohen's d results show small practical differences in measurements between studies and between researchers.The three highest values of the order of 0.343; 0.359; 0.373 demonstrate a low-moderate effect size.The remainder are at a weak level.Referring to Ellis (2010) norms: 0.2-weak effect; 0.5-moderate effect; 0.8-strong effect.
The present study was intended as the next step in investigating the effectiveness of Luna EMG, with a focus on patients with stroke, a neurological condition adversely affecting proprioception.Research shows that approximately 50% of individuals after a stroke experience upper limb proprioception deficits (He et al., 2022;Meyer et al., 2016), which negatively affects functional performance (Ingemanson et al., 2019), activity and participation in daily life (Carey, Matyas & Baum, 2018) and motor control (Carlsson, Gard & Brogårdh, 2018).The present study was conducted in a group of 126 patients with chronic stroke.The sample size is a strength of the study.The number of participants enrolled is representative, and the sample size was calculated using a statistical research   planning method.The authors decided to focus on patients with chronic stroke because research shows that the most pronounced progress in the recovery of neuromotor function is observed at early stages post-stroke (Duncan, Lai & Keighley, 2000;Lee et al., 2015) due to the fact that changes associated with regeneration within the penumbra occur relatively quickly after stroke onset and the process slows down in the later stages (Dąbrowski et al., 2019;Kopp et al., 1990).Therefore, the study took into account patients at a chronic stage post-stroke who may have already adapted to the use of the persistent motor pattern in the affected upper limb and have developed compensatory strategies in the unaffected limb.
Another reason for undertaking this research lies in the fact that very few studies so far have investigated upper limb proprioception in patients with stroke (Rand, 2018;Kiper et al., 2015;Ocal, Alaca & Canbora, 2020); however, the findings reported are rather promising.As an example, Kiper et al. (2015) demonstrated that proprioceptive training may lead to improvements in patients with upper limb paralysis after subacute stroke.Furthermore, a study by Ocal, Alaca & Canbora (2020) showed that, compared to a conventional therapy, proprioceptive training of the upper limb more effectively increases the frequency and quality of movements performed with the upper limb by patients at a chronic stage post-stroke.Therefore, we believe that it is necessary to continue research focusing on various issues associated with upper limb proprioception post-stroke.Our findings show that the Luna EMG multifunction robotic device is a reliable tool in the evaluation of upper limb proprioception in patients with chronic stroke.It is a well-established fact that diagnostic assessments are essential for proper planning and monitoring of training-based therapies (Sarzy nska-Długosz, 2023); therefore, in further studies we intend to evaluate the effectiveness of Luna EMG in assessing the effects of upper limb proprioceptive training in patients after stroke.
In summary of these considerations we can say that this study presents the first scientific evidence related to evaluation of upper limb proprioception in patients with chronic stroke, performed using the Luna EMG multifunction robotic device, and the findings are consistent with results of other studies involving healthy participants, and they demonstrate high reliability of the tool in evaluating upper limb proprioception in patients at a chronic phase after stroke.

Limitations
The study has some limitations, most importantly, evaluation of upper limb proprioception performed with the Luna EMG diagnostic and rehabilitation device was not supported with results acquired using proprioception assessment scales and tests.However, we are planning further research in which effects of stroke rehabilitation will be evaluated using Luna EMG as well as assessment scales and tests.

CONCLUSIONS
The study demonstrates that the Luna EMG multifunction robotic device is a reliable tool in the evaluation of upper limb proprioception.Measurements made with Luna EMG show high inter-rater and intra-rater agreement in the assessment of the proprioceptive sensation of the upper limb in patients with stroke. .Anna Roksela conceived and designed the experiments, prepared figures and/or tables, and approved the final draft. .Agnieszka Guzik conceived and designed the experiments, performed the experiments, analyzed the data, authored or reviewed drafts of the article, and approved the final draft.

Human Ethics
The following information was supplied relating to ethical approvals (i.e., approving body and any reference numbers): The University of Rzeszow (resolution no.2022/036/W) approved the study.

Table 2
Agreement between measurements of right hand proprioception.
Note:x, mean; SD, standard deviation; , degrees of angle; p, assessment of the significance of differences in the average level of two series of measurements (t-test for related samples); d, wielkość effect size; d, Cohena; r, Pearson's linear correlation coefficient; ICC, intraclass correlation coefficient with 95% confidence interval; CV, coefficient of variation; SEM, standard error of the mean.

Table 3
Agreement between measurements of left hand proprioception.SD, standard deviation; , degrees of angle; p, assessment of the significance of differences in the average level of two series of measurements (t-test for related samples); d, wielkość effect size; d, Cohena; r, Pearson's linear correlation coefficient; ICC, intraclass correlation coefficient with 95% confidence interval; CV, coefficient of variation; SEM, standard error of the mean.